AI for art history

Barbaric rituals and bizarre oaths are said to have accompanied Claudius Civilis’ conspiracy in the first century of the Christian era, when the ruler led an uprising against Rome. Dutch painter Rembrandt van Rijn captured one such scene on canvas. Statistician and art historian Stefanie Schneider uses precisely this painting to explain how many different aspects come into play when comparing different works. How is the situation depicted, for example? With what materials did the artist work? And in what historical context should the image be seen? To draw on the capabilities of artificial intelligence in answering such questions in the future, the project “Reflection-driven Artificial Intelligence in Art History” was recently launched at the LMU’s Institute of Art History.

A team headed by Professor Hubertus Kohle, Chair of Medieval and Modern Art History at LMU, and Professor Ralph Ewerth, Head of the Visual Analytics research group at the German National Library of Science and Technology (TIB) in Hanover, is working on developing an AI tool. The AI is being trained with the art history knowledge it needs to serve as a useful instrument to support analysis of similarities between images in the context of art history. The project is funded under the aegis of the German Research Foundation (DFG) priority program “The Digital Image”.

The interdisciplinary team brings together researchers from the world of art history and informatics. Their shared goal is to develop an algorithm whose decision process is clear and intuitive. “We want the process that AI uses to reach its findings to be transparent. We want to visualize it,” says Stefanie Schneider, research assistant for digital art history at LMU’s Chair of Medieval and Modern Art History.

The first step is to examine historical scientific methods of automated image analysis and build a data library of images and texts in order to train the algorithm. For example, the researchers use freely accessible metadata from museums and from images for which copyright protection has already expired (as the copyright holders have been dead for at least 70 years). To prevent the AI system from reproducing prejudices – for example because fewer female than male artists are represented in the training data – search results flag any possible biases and the reasons for them. Another aspect here involves analyzing how certain text resources affect the results.

By transparently depicting the methodologies used, the project aims to encourage acceptance of the use of AI in the study of the arts. “AI is not going to do away with art historians,” Schneider insists. “The system will make suggestions that must be validated by academics in the context of art history.”

Other research associates involved in the project are Julian Stalter and Maximilian Kristen (both LMU), alongside Matthias Springstein and Eric Müller-Budack (both TIB Hanover).

For more information on the project, see:

The digital image: Reflect AI